1. Field of the Invention
The embodiments of the invention provide a method, computer program product, etc. for risk-modulated proactive data migration for maximizing utility.
2. Description of the Related Art
Growing consolidation of storage systems necessitates resource sharing among multiple competing applications with different access characteristics and Service Level Objectives (SLOs). The goal of storage management is to allocate the resources to each application such that the number of SLOs satisfied is maximized. The decision-making for resource allocation is not a one time task, but rather an ongoing process due to the existence of workload variations, application changes, system exceptions (failures and load surges), the resource to application mapping is not a static one-time task and administrators often trigger corrective actions to adjust resource allocation dynamically.
Migration is one of the commonly used corrective actions—it changes the resources allocated to a specific application. Existing commercial tools help in the decision-making for what dataset to migrate and where to migrate. Also, there is ongoing research on deciding the migration speed using feedback loop.
However, migrating data in large scale storage systems that are always full with a continuously high load has additional challenges not addressed by existing research. First, the decision for when to start migration must be made. It has been commonly assumed that migration is triggered by the administrators, typically when the system is lightly loaded (for example, at night) or reactively when problem happened (mostly as background task). The decision for when to start migration is nontrivial since it needs to take account the workload trends, current impact on the utility of applications, etc. Additionally, migration needs to be planned in advance since moving terabytes of data can take days or weeks. In summary, automatically deciding when to start migration action must be considered.
Furthermore, existing research makes migration decisions based on the “current” system state—there is a lack of consideration for the long-term temporal behavior leading to sub-optimal solutions and wastage of system resources (in moving unnecessary data around) or failure to prevent resource contention problems proactively (resulting in more SLO violations than desired).
Additionally, there is always a certain amount of risk involved in moving data—the models for workload growth may have a high volatility or a transient overload may be misunderstood as a permanent load pattern. Most migration tools simply aim to maximize the storage utility—in reality it is required to maximize utility with minimal risks.